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Abstract 



> 

Agents are small programs that autonomously take actions based on 
^^ ' changes in their environment or "state." Over the last few years, there 

(_^ I have been an increasing number of efforts to build agents that can interact 



and/or collaborate with ot her agents, fn one of these efforts, Eiter, Sub 



rahmanian, and Pick (1999) have shown how agents may be built on top of 



i^ , legacy code. However, their framework assumes that agent states are com- 

C/2 ■ pletely determined, and there is no uncertainty in an agent's state. Thus, 

^ ' their framework allows an agent developer to specify how his agents will 

react when the agent is 100% sure about what is true/false in the world 
, J. state. In this paper, we propose the concept of a probabilistic agent pro- 

kS [ gram and show how, given an arbitrary program written in any imperative 

language, we may build a declarative "probabilistic" agent program on top 
of it which supports decision making in the presence of uncertainty. We 
provide two alternative semantics for probabilistic agent programs. We 
show that the second semantics, though more epistemically appealing, is 
more complex to compute. We provide sound and complete algorithms to 
compute the semantics of positive agent programs. 



*Most proofs are contained in the appendix. 
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1 Introduction 

Over the last few years, there has been increasing interest in the area of soft- 
ware agents. Such agents provide a wide variety of services including identi- 
fication of interesting newspaper articles, software robots that perforin tasks 
(and plan) on a user's behalf, content based routers, agent based telecommu- 
nication applications, and solutions to log istics applications. IMPACT (see 



I http://www.cs.uind.edu/projects/iinpact/ — ) is a multinational project whose 



aim is to define a formal theory of software agents, implement (appropriate 
fragments of) the theory efficiently, and develop an appropriate suite of appli- 
cations on top of this implementation. An IMPACT agent manages a set of 
data types/structures (including a message box) through a set of application 
program interface (API) function calls. The state of the agent at a given point 
in time is a set of objects belonging to these data types. Each agent has a set of 
integrity constraints that its state must always satisfy. When an agent's state 
changes (due to external events such as receipt of a message) , the agent tries to 
modify its state so that the integrity constraints are satisfied. To do this, it has 
a suite of actions, and an agent program that specifies the operating principles 
(what is permitted, what is forbidden, what is obligatory, etc., and under what 



conditions?). (Eiter, Subrahmanian, and Pick 1999; Eiter and Subrahmanian 



1999) provides a detailed study of the semantics and complexity of such agents. 



(Eiter, Subrahmanian, and Rogers 199£) contains compile-timc and run-time 



algorithms, while (Arisha, Ozcan, Ross, Subrahmanian, Eiter, and Kraus 1999) 



focuses on system architecture. 

Past work on IMPACT assumes that all agents reason with a complete and 
certain view of the world. However, in many real world applications, agents 
have only a partial, uncertain view of what is true in the world. Though an 
agent may need to reason about uncertainty for many reasons, in this paper, 
we will assume that the main cause of uncertainty in an agent is due to its 
state being uncertain. For example, when an image processing agent is asked 
to identify an enemy vehicle, it might return the fact that vehicle vi is a T72 
tank (with 60-70% probability) and a T-80 tank (with 20-45% probability). 
However, this raises several problems, the first of which is that as an action 
can only be executed if its precondition is true in the current state, if the agent 
doesn't know what the state is, then it cannot determine which of its actions 
are executable, and which are not. Second, even if an action is executable, the 
state that results may not be precisely determinable either. One consequence of 
all this is that the semantics of agent programs change significantly when such 
uncertainties arise. 

The main contributions (and organization) of this paper may now be summed 
up as follows. 

1. In Section 0, we present a brief overview of agents (without any uncertainty 



involved) as described in (Eiter, Subrahmanian, and Pick 1999) 



2. Then, in Section pi we define the concept of a probabilistic code call, which 
is the basic syntactic construct through which uncertainty in abstract data 



types manifests itself. 

3. In Section 0, we define the syntax of probabilistic agent programs. Specif- 
ically, we show that probabilistic agent programs allow an agent developer 
to specify the permissions, obligations, forbidden actions, etc. associated 
with an agent depending not only on the probabilities that certain con- 
ditions hold in the agent's state, but also on the developer's assumptions 
about the relationship between these conditions (e.g. the probability that 
a conjunction holds in a given state depends not only on the probabilities 
of the conjuncts involved, but also on the dependencies if any between the 
conjuncts). 

4. In Section |^, we develop three formal semantics for probabilistic agent 
programs which extend each other as well as t he semantics for (ordinary, 
non probabilistic) agent programs defined by Eiter, Subrahmanian, anq 



Pick (1999 ). We also provide results relating these diverse semantics. 

5. Then, in Section^, we develop a sound and complete algorithm to compute 
the semantics defined when only positive agent programs are considered. 



We also show that the classical agent programs of Eiter, Subrahmanian, 
and Pick (1999|) are a special case of our probabilistic programs. 



6. In Section R we provide an alternative, Kripke style semantics for agent 
programs. In contrast to the previous "family" of semantics which assume 
that an agent's precondition must be true with 100% probability for the 
agent to execute it, this semantics also allows an agent to execute it when 
it is not sure (with 100% probability) that the action's precondition is 
true. We extend all three semantics of agent programs defined earlier in 
Section to handle these intuitions. Unfortunately, as we show in this 
section, this desire for a "more sophisticated" sematics comes at a high 
computational price. 

2 Preliminaries 

In IMPACT, each agent a is built on top of a body of software code (built in any 
programming language) that supports a well defined application programmer 
interface (cither part of the code itself, or developed to augment the code). 
Hence, associated with each agent a is a body of software code S'^ defined as 
follows. 

Definition 2.1 (Software Code) We may characterize the code on top of 
which an agent is built as a triple S —def {Tst^StCs) where: 

1. Ts is the set of all data types managed by S, 

2. Ts is a set of predefined functions which makes access to the data objects 
managed by the agent available to external processes, and 



3. Cs is a set of type composition operations. A type composition operator 
is a partial n-ary function c which takes types ri,... ,t„ as input, and 
yields a type c(ri, . . . ,t„) as output. As c is a partial function, c may 
only be defined for certain arguments ti, . . . ,t„, i.e., c is not necessarily 
applicable on arbitrary types. 

When Q is clear from context, we will often drop the superscript a. Intuitively, 
Ts is the set of all data types managed by a, Ts is the set of all function 
calls supported by iS's application programmer interface (API). Cs is the set of 
ways of creating new data types from existing data types. This characterization 
of a piece of software code is widely used (cf. the Object Data Management 
Group's O DMG standa rd dCattcU, R. G. G., et al. 1997| ) and the CORBA 
framework ( Siegal 1996 )). Each agent also has a message box having a well 
defined set of associated code calls that can be invoked by external programs. 



Example 2.2 [Surveillance Example] Consider a surveillance application where 
there are hundreds of (identical) surveillance agents, and a geographic agent. 
The data types associated with the surveillance and geographic agent include 
the standard int ,bool,real, string, file data types, plus those shown below: 



Surveillance Agent 



Geographic Agent 



imageirecord of 

imageid:file; 
day:date; 
time: int; 
location:string 
imagcdb: setof image; 



map: I quadtree; 
quadtree:record of 

place:string; 

xcoord:int; 

ycoord:int; 

pop:int 

nw,ne,sw,se:t quadtree 



A third agent may well merge information from these two agents, tracking a 
sequence of surveillance events. 

The surv agent may support a function surv : identify{) which takes as input, 
an image, and returns as output, the set of all identified vehicles in it. It may 
also support a function called surv : turret{) that takes as input, a vehicle id, and 
returns as output, the type of gun-turret it has. Likewise, the geo agent may 
support a function geo : getplnodei) which takes as input a map and the name 
of a place and returns the set of all nodes with that name as the place-field, 
a function geo : getxynodeO which takes as input a map and the coordinates 
of a place and returns the set of all nodes with that coordinate as the node, 
a function called geo :range{) that takes as input a map, an x,y coordinate 
pair, and a distance r and returns as output, the set of all nodes in the map 
(quadtree) that are within r units of location {x,y). 

Throughout this paper, we will expand on this simple example and use it to 
illustrate and motivate the various definitions in the paper. 



loci 50.50 



Loc 4 52, GO 



Loc2 55. 40 
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Loc 3 5 3,45 





Loc5 37.42 



Figure 1: Example quadtree for Surveillance Application. 

Definition 2.3 (State of an Agent) The state of an agent at any given point 
t in time, denoted Os{t), consists of the set of all instantiated data objects of 
types contained in Tg . 

An agent's state may change because it took an action, or because it received 
a message. Throughout this paper we will assume that except for appending 
messages to an agent q's mailbox, another agent b cannot directly change a's 
state. However, it might do so indirectly by shipping the other agent a message 
requesting a change. 

Example 2.4 For example, the state of the Geographic Agent may consist of 
two quadtrees (one of which, mapl, is shown in Figure |l|), and the type "map" 
may contain two objects, mapl, and niap2, pointing to these two quadtrees, 
respectively. (The figure doesn't show population values explicitly. Assume the 
population values are 20,000 for Loci, 28,000 for Loc2, 15,000 for Loc3, and 
40,000 for Loc4, and 8000 for Loc5.) 



Queries and/or conditions may be evaluated w.r.t. an agent state using the 
notion of a code call atom and a code call condition defined below. 



Definition 2.5 (Code Call/Code Call Atom) // S is the name of a soft- 
ware package, f is a function defined in this package, and {di, . . . ,d„) is a 
tuple of arguments of the input type of f, then S ;/(di, . . . , dn) is called a code 
call. 

// cc is a code call, and X is either a variable symbol or an object of the 
output type of cc, then in(X, cc) is called a code call atom. 

For instance, in the Surveillance example, geo : getplnode {majpl, "Loci") returns 
the set containing just the single node referring to Loci in Figure n^. Likewise, 
the code call geo : range{ma.pl, 55, 50, 11) returns the set containing the nodes 
labeled Loci and Loc2. 



Definition 2.6 (Code Call Condition) A code call condition x is defined as 
follows: 

1. Every code call atom is a code call condition. 

2. If s,t are either variables or objects, then s = t is a code call condition. 

3. If s,t are either integers/real valued objects, or are variables over the 
integers/reals, then s<t,s>t,s>t,s<t are code call conditions. 

4- If X1tX2 o,re code call conditions, then X1&X2 is a code call condition. 

A code call condition satisfying any of the first three criteria above is an atomic 
code call condition. 

An example code call condition is shown below. 

Example 2.7 in{X, geo :range{ma.-pl, 55, 50,11)) SzX.pop > 25,000 is a code 
call condition that is satisfied by only one node in mapl, viz. the Loc2 node. 

Each agent has an associated set of integrity constraints — only states that 
satisfy these constraints are considered to be valid or legal states. An integrity 
constraint is an implication whose consequent is a code call atom, and whose 
antecedent is a code call condition. Appendix A contains a detailed definition. 
Each agent has an action-base describing various actions that the agent is 
capable of executing. Actions change the state of the agent and perhaps the 
state of other agents' msgboxes. As in classical AI, all actions have an associated 
precondition (a code call condition that the agent state must satisfy for the 
action to be executable) and an add/delete list. Appendix A contains detailed 
definitions from (Eiter, Subrahmanian, and Pick 1999|). 



For instance, the geo agent may have an insert action that adds a node to 
the map. Likewise, the surv agent may also have an insert action which inserts 
a new image into the image database. Both these agents also have an action 
that sends a message. 

Each agent has an associated "notion of concurrency," cone, which a set 
of actions and an agent state as input, and produces as output, a single action 
that reflects the combination of all the input actions. ( Eiter, Subrahmanian, ancj 



Pick 1999 ) provides examples of three different notions of concurrency. We will 
sometimes abuse notation write conc(S', O) to denote the new state obtained 
by concurrently executing the actions in S in state O. 

Each agent has an associated set of action constraints that define the cir- 
cumstances under which certain actions may be concurrently executed. As at 
any given point t in time, many sets of actions may be concurrently executable, 
each agent has an Agent Program that determines what actions the agent can 
take, what actions the agent cannot take, and what actions the agent must take. 
Agent programs are defined in terms of status atoms defined below. 



Definition 2.8 (Status Atom/Status Set) // a(i) is an action, and Op S 
{P, F, W, Do , O}, then Opa{t) is called a status atom. If A is an action status 
atom, then A, -lA are called status literals. A status set is a finite set of ground 
status atoms. 

Intuitively, Pa means a is permitted, Fa means a is forbidden, Do a means a 
is actually done, and Wa means that the obligation to perform a is waived. 

Definition 2.9 (Agent Program) An agent program V is a finite set of rules 
of the form 

A ^ x^Lik...kLn 
where x is a code call condition and Li, . . . , L„ are status literals. 



The semantics of agent programs are well described in (Eiter, Subrahmanian, 



and Pick 199£ ; Eiter and Subrahmanian 1999 ) — due to space reasons, we do not 
explicitly recapitulate them here, though Appendix ^ contains a brief overview 
of the semantics. 

3 Probabilistic Code Calls 

Consider a code call of the form d:/(args). This code call returns a set of 
objects. If an object o is returned by such a code call, then this means that 
o is definitely in the result of evaluating d:/(args). However, there are many 
cases, particularly in applications involving reasoning about knowledge, where 
a code call may need to return an "uncertain" answer. In our our surveillance 
example, surv : identify {imaigel) tries to identify all objects in a given image — 
however, it is well known that image identification is an uncertain task. Some 
objects may be identified with 100% certainty, while in other cases, it may only 
be possible to say it is either a T72 tank with 40-50% probability, or a T80 
tank with (50-60%) probability. 

Definition 3.1 (Random Variable of Type r) A random variable of type 
T is a finite set RV of objects of type t , together with a probability distribution 
p that assigns real numbers in the unit interval [0, 1] to members of RV such 
that SogRvpIo) < 1. 



It is important to note that in classical probability theory ( Ross 1997 ), random 
variables satisfy the stronger requirement that EogRvp(o) = 1. However, in 
many real life situations, a probability distribution may have missing pieces, 
which explains why we have chosen a weaker definition. However, the classi- 
cal probability case when EogRvp(o) = 1 is an instance of our more general 
definition. 

Definition 3.2 (Probabilistic Code Call Q:Rv/(di,... ,dn)) 

Suppose a :/(di, . . . , d^) is a code call whose output type is r. The probabilistic 
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code call associated with a:/(di,... ,dn), denoted a:R,v/(di, . . . ,dn), returns 
a set of random variables of type t when executed on state O. 

The following example illustrates the use of probabilistic code calls. 

Example 3.3 Consider the code call surv: identify (ima^gel) . This code call 
may return the following two random variables. 

({t72, t80}, {(i72, 0.5), (t80, 0.4)}) and ({i60, t84}, {(i60, 0.3), (t84, 0.7)}) 

This says that the image processing algorithm has identified two objects in 
imagel. The first object is cither a T72 or a T80 tank with 50% and 40% 
probability, respectively, while the second object is either a T60 or a T84 tank 
with 30% and 70% probability respectively. 

Probabilistic code calls and code call conditions look exactly like ordinary code 
calls and code call conditions — however, as a probabilistic code call returns 
a set of random variables, probabilistic code call atoms are true or false with 
some probability. 

Example 3.4 Consider the probabilistic code call condition 

in(X, surv :rv identify {im&gel)) &in(Al, surv :rv turret{X)). 

This code call condition attempts to find all vehicles in "imagel" with a gun 
turret of type Al. Let us suppose that the first code call returns just one 
random variable specifying that imagel contains one vehicle which is either a 
T72 (probability 50%) or a T80 tank (probability 40%). When this random 
variable (X) is passed to the second code call, it returns one random variable 
with two values — Al with probability 30% and A2 with probability 65%. What 
is the probability that the code call condition above is satisfied by a particular 
assignment to X? 

The answer to this question depends very much upon the knowledge we have 
(if any) about the dependencies between the identification of a tank as a T-72 
or a T-80, and the type of gun turret on these. For instance, if we know that all 
T72's have A2 type turrets, then the probability of the conjunct being true when 
X is a T72 tank is 0. On the other hand, it may be that the turret identification 
and the vehicle identification are independent for T80s — hence, when X is set 
to T80, the probability of the conjunct being true is 0.4 x 0.3 = 0.12. 

Unfortunately, this is not the only problem. Other problems also arise, as shown 
in the following example. 

Example 3.5 Suppose we consider a code call x returning the following two 
random variables. 

RVi = ({a,6},pi) 
RV2 - ({6,c},p2) 



Suppose pi (a) = 0.9, pi{b) =0.1, p2{b) — 0.8, p2(c) =0.1. What is the proba- 
bihty that 6 is in the resuh of the code caU x? 

Answering this question is problematic. The reason is that we are told that 
there are at most two objects returned by x- One of these objects is either a or b, 
and the other is either b or c. This leads to four possibilities, depending on which 
of these is true. The situation is further complicated because in some cases, 
knowing that the first object is b may preclude the second object from being 
b — this would occur, for instance, if x examines photographs each containing 
two different people and provides identifications for each, a, b and c may be 
potential id's of such people returned by the image processing program. In such 
cases, the same person can never be pictured with himself or herself. 

Of course, in other cases, there may be no reason to believe that knowing 
the value of one of two objects tells us anything about the value of the second 
object. For example if we replace people with colored cubes (with a denoting 
amber cubes, b black, and c cyan), there is no reason to believe that two identical 
black cubes cannot be pictured next to each other. 

One could argue, however, that the above reasoning is incorrect because if two 
objects are completely identical, then they must be the same. This means 
that if we have two distinct black cubes, then these two black cubes must be 
distinguishable from one another via some property such as their location in 
the photo, or their Ids. This is Leibniz's well known extensionality principle. 
Hence, we will require the results of a probabilistic code call to be coherent in 
the following sense. 

Definition 3.6 (Coherent Probabilistic Code Call) A probabilistic code call 
is coherent iff for all distinct {Xi, pi), {X2, P2), X\ n X2 = 0. 

Throughout this paper, only coherent probabilistic code calls are considered. 
Thus, the expression "probabilistic code call" assumes coherence. 

Definition 3.7 (Satisfying a Code Call Atom) Suppose a:Hv/(di, . . . , dn) 
is a ground probabilistic code call and o is an object of the output type of this 
code call w.r.t. agent state O. Suppose [i,u] is a closed subinterval of the unit 
interval [0, 1]. 



• 



• 



o |=|^'"1 in(X, a:Rv/(di,... ,dn)) 

if there is a {Y, p) in the answer returned by evaluating a :Rv/(di, . . . , d^) 

w.r.t. O such that Cz Y and £ < p{o) < u. 

^1^'"' not_in(X, a:Rv/(di,. .. ,dn)) 

if for all random variables (y, p) returned by evaluating a :Rv/(di, . . . , dn) 

w.r.t. O, either ^Y or p{o) ^ [(-,u\. 

Probabilistic code call conditions are defined in exactly the same way as code 
call conditions. However, extending the above definition of "satisfaction" to 
probabilistic code call conditions is highly problematic because (as shown in 



Examples 3.4 and |3.5D , the probability that a conjunction is true depends not 
only on the probabilities of the individual conjuncts, but also on the dependen- 
cies between the events denoted by these conjuncts. The notion of a probabilistic 
conjunction strategy defined below captures these different ways of computing 
probabilities via an abstract definition. 

Definition 3.8 (Probabilistic Conjunction Strategy 0) 

A probabilistic conjunction strategy is a mapping Cg) which maps a pair of prob- 
ability intervals to a single probability interval satisfying the following axioms: 

1. Bottomline: [Li,Ui]®[L2,U2] < [min(ii, L2),min(C/i, [/2)] where[x,y] < 
[x',y'] if X < x' and y < y' . 

2. Ignorance: [Li, Ui] ® [L2, U2] C [max(0, Li + L2 — 1), min(C/i, 1/2)]- 

3. Identity: When (ei A 62) is consistent and [L2,C^2] — [1,1], [-^i,C^i] ® 
[L2,U2] = [Li,Ui]. 

4. Annihilator: [Li,Ui] (g) [0,0] = [0,0]. 

5. Commutativity: [Li, Ui] [L2, U2] = [£2, U2] <E) [Li, Ui]. 

6. Associativity: {[Li.Ui] ® [L2,U2]) ® [Lz^U^i] = [Li,C/i] ® {[L2,U2] ® 

7. Monotonicity: [Li, t/i] ® [L2, C/2] < [Li, C/i] ® [L3, i/g] if [L2,U2] < 

Intuitively, in the above definition, [Li, Ui], [L2, U2] are intervals in which the 
probability of events 61,62 are known to lie, and [Li,C/i] (g) [L2,U2] returns a 
probability range for the co-occurrence of both these events. The Bottomline 
axiom says that the probability of the conjunct is smaller than the probabilities 
of the individual events. When we know nothing about the relationship between 



the events 61,62, Boole (1854) has shown that the probability of the conjunc- 
tion must lie in the interval [max(0,Li + L2 — l),min(C/i, [/2)]- This is what 
is stated in the Ignorance axiom. The identity and annihilator axioms specify 
what happens when one of the events is deterministic (i.e. not probabilistic). 
The axioms of commutativity and associativity are self explanatory. The mono- 
tonicity axiom says that if we sharpen the probability range of one of the two 
events, then the probability range of the conjunctive event is also sharpened. 

The concept of a conjunction strategy is very general, and has as special 
cases, the following well known ways of combining probabilities. 

1. When we do not know the dependencies between 61,62, we may use the 
conjunction strategy (E)ig defined as ([Li, Ui] ®ig [L2, U2]) = [max(0,Li -|- 
L2-l),min(C/i,[/2)]. 

2. When 61, 62 have maximal overlap, use the positive correlation conjunctive 
strategy (^pc defined as ([Li, Ui](^pc[L2, U2]) = [min(Li, L2), min([/i, C/2)]. 
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3. When 61,62 have minimal overlap, use the negative correlation conjunc- 
tive strategy ^nc defined as ([Li,t/i] ®„c [L2,U2]) = [max(0, Li + L2 ~ 
l),max(0,C/i + C/2-l)]. 

4. When the two events occur independently, use the independence conjunc- 
tion strategy {[Li, Ui] (g)i„ [L2, U2]) = [Li ■ L2, Ui ■ C/2]. 

4 Probabilistic Agent Programs: Syntax 

We are now ready to define the syntax of a probabilistic agent program (pap 
for short). This syntax builds upon the well studied annotated logic paradigm 



proposed by( 3ubrahmanian 1987), and later studied extensively (Kifer and Sub 



rahmanian 1992; Ng and Subrahmanian 1993b; Ng and Subrahmanian 1993a) 



4.1 Annotation Syntax 

We assume the existence of an annotation language L°"" — the constant symbols 
of L°"" are the real numbers in the unit interval [0,1]. In addition, L™" 
contains a finite set of function symbols, each with an associated arity, and 
a (possibly infinite) set of variable symbols, ranging over the unit interval [0, 1]. 
All function symbols are pre-interpreted in the sense that associated with each 
function symbol / of arity A: is a fixed function from [0, 1]'' to [0, 1]. 

Definition 4.1 (Annotation Item) We define annotation items inductively 
as follows: 

• Every constant and every variable of L""" is an annotation item. 

• If f is an annotation function of arity n and aii, . . . ,ai„ are annotation 
items, then /(aii, . . . ,ai„) is an annotation item. 

An annotation item is ground if no annotation variables occur in it. 

For instance, 0, 0.9, (F-l-0.9), (^-(-0.9)^ are all annotation items if y is a variable 
in L™" and "-I-", "" are annotation functions of arity 2. 

Definition 4.2 (Annotation [aii,ai2]) //aii,ai2 are annotation items, then 
[aii,ai2] is an annotation. //aii,ai2 are both ground, then [aii,ai2] is a ground 
annotation. 

For instance, [0, 0.4], [0.7, 0.9], [0.1, -j], [-j, -j] are all annotations. The annota- 
tion [0.1, -j] denotes an interval only when a value in [0, 1] is assigned to the 
variable V. 

Definition 4.3 (Annotated Code Call Condition x '■ ([aii, ai2], (8>)) Ifx '^^ 
a probabilistic code call condition, (i^ is a conjunction strategy, and [aii,ai2] 
is an annotation, then x '■ ([aii, ai2], ®) *s an annotated code call condition. 
X '■ ([ail, ai2], 0) is ground if there are no variables in either x or in [aii,ai2]. 
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Intuitively, the ground annotated code call condition x '■ ([aii, ai2], (8>) says that 
the probability of x being true (under conjunction strategy ®) lies in the interval 
[aii,ai2]. For example, when X,A\ are ground, 

in(X, surviRv irfenii!/?/(imagel)) &in(Al, survtRv turret{X)) : ([0.3, 0.5], (Xi^g) 

is true if and only if the probability that X is identified by the surv agent and 
that the turret is identified as being of type Al lies between 30 and 50% assuming 
that nothing is known about the dependencies between turret identifications and 
identifications of objects by surv. 

We are now ready to define the concept of a probabilistic agent program. 

Definition 4.4 (Probabilistic Agent Programs PV) Suppose T is a con- 
junction of annotated code calls, and A, Li,... ,L„ are action status atoms. 
Then 

A^r,Li,... ,L„ (1) 

is a probabilistic action rule. For such a rule r, we use B^g(r) to denote the 
positive action status atoms in {Li, . . . ,Ln}, and B^g{r) to denote the set of 
negative action status liters in {ii, . . . , L„}. 

A probabilistic agent program is a finite set of probabilistic action rules. 

A simple example of a probabilistic agent program is given below. 

Example 4.5 [Probabilistic Agent Program] Consider an intelligent sensor agent 
that is performing surveillance tasks. The following rules specify a small pap 
that such an agent might use. 



Do send_warn{X) ^- in(F, surv : /i^e(imagedb)) & 

in(X, surv :rv identify{F)) & 
in(Al, surv :rv turret{t))) : ([0.7, l.%®ig) 
-iF sendjwarn{X). 
Fsend-warn{X) ^- in{X, surv:-Rv identify {¥)) &: 

in(L, geo :rv getplnode{X.locat±oTL)) & 
in(L, geo :rv range{100, 100, 20)). 

This agent operates according to two very simple rules. The first rule says 
that it sends a warning whenever it identifies an enemy vehicle as having a gun 
turret of type Al with over 70% probability, as long as sending such a warning 
is not forbidden. The second rule says that sending a warning is forbidden if 
the enemy vehicle is within 20 units of distance from location (100,100). 
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5 Probabilistic Agent Programs: Semantics 

We are now ready to define the semantics of paps. Tfie semantics of paps will 
be defined via the concept of a probabilistic status set (defined below). 

Definition 5.1 (Probabilistic Status Set VS) A probabilistic status set is 
any set VS of ground action status atoms over S. For any operator Op € 
{P,Do,F, 0,W}, we denote by Op{VS) the set {a \ Op{a) G VS}. Similarly, 
we use —VS to denote the set {^A \ A G VS}. 

It will turn out that given any probabilistic agent program, and an (uncertain) 
agent state evaluated using probabilistic code calls, the meaning of the pap w.r.t. 
the state may be defined via a set of probabilistic status sets that have some 
desirable properties. These properties fall into three broad categories: 

1. the probabilistic status set must be "closed" under the rules in the pap; 

2. the probabilistic status set must be deontically consistent (e.g. it cannot 
require something to be both permitted and forbidden) and it must not 
violate the action constraints; 

3. the probabilistic status set must not lead to a new state that violates the 
integrity constraints associated with the agent; 

5.1 Satisfaction of Annotated Formulae 

In this section, we define what it means for an agent state to satisfy an annotated 
code call condition. 

Definition 5.2 (Satisfying an Annotated Code Call Condition) Suppose 
O is an agent state, and x ■ {[^h, ai2], ®) is a ground annotated code call condi- 
tion. O is said to satisfy x ■ ([aii, ai2], i?)), denoted O ^[^'i^^'^l t^ : ([aii, ai2], (8)) 
iff-- 

• X is of the form a ~ o (where o is an object), or 

• X is of the form ri < r2, where ri,r2 are real numbers (or integers) such 
that ri is less than r2, or 

• Xis of the form in{X, Q:Rv/(di,... ,dn)) and a |==g"'^"' in(X, a:Rv/(di,... ,d^)), 
or 

• X is of the form not_in(o, a :Rv/(di, . . . ,dn)) and the following holds 

Jail 



o l^g"'"'^' not_in(X, a :Rv/(di, . . . , dn)), or 



• X is of the form xi A X2 o,nd [i?i,wi], [£2,W2] are the tightest intervals such 
that O I^^I'^i^"!] xi and O I^I'^^.M ^^ and [aii,ai2] 3 [ii,ui] (g) [^2,U2]. 

O is said to satisfy a non-ground annotated code call x '■ ([aii, ai2], ®) iff O 
satisfies all ground instances of x '■ ([aii, ai2], (8)). 
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5.2 Closure and App-p-p^Osi'PS) 

We may associate with any pap W, an operator App-p-pQ^CPS) which maps 
probabihstic status sets to probabihstic status sets. 

Definition 5.3 (Operator App-p-p ^0^(1^3)) Suppose W is a probabilistic agent 
program, Os is an agent state, and VS is a probabilistic status set. Then 

App-p-pQg{PS) — {Op a I Op a is the head of a ground instance r of a rule 

in VP satisfying the 4 conditions below } 

1. B+^{r) C VS and -.^-^(r) r\VS = %, and 

2. For every annotated code call condition x '■ ([ski 3i2]j 'X') in the body of r, 
it is the case that Os |=[^'i'^'2l ^ : ([aii, ai2], (Xi) and 

3. if Ope {P,0,Do}, then Os h'^'^' Pre{a) and 

4- for every action status atom of the form Op (3 in B^^ir) such that Op € 
{P,0,Do}, Osh^^-^^ Pre{l3). 

The first part of this definition says that for a rule to fire, the action status atoms 
in its body must be "true" w.r.t. VS. The second condition says that annotated 
code call conditions in a rule body must be satisfied in the current object state 
for the rule to fire. The third part is more tricky. It says that if Oa or Do a 
or Pa is in the head of a rule, then for the rule to fire, the precondition of the 
action must be true with 100% probability. The final condition is similar w.r.t. 
to positive action status atoms in the body. Thus, for now, we are assuming that 
for an agent to perform an action (or even be permitted to perform an action), 
it must be 100% sure that the action's precondition is true (later in Section pi we 
will provide an alternate, more complex semantics that does not require this). 

Definition 5.4 (Closure under Program Rules) VS is said to be closed 
under the rules of pap VV in state O^ iff App-p-pQ^iVS) C VS. 

5.3 Deontic/Action Consistency/Closure 

The concept of deontic/action consistency requires that probabilistic status sets 
satisfy the agent's action constraints and commonsense axioms about deontic 
modalities. 

Definition 5.5 (Deontic and Action Consistency) A probabilistic status set 
VS is deontically consistent with respect to an agent state O iff it satisfies the 
following rules for any ground action a: 

• IfOae VS, then Wa ^ VS. 

• //Pa e VS, then Fa ^ VS. 

• //Pa e VS, then O h'^'^' Pre{a). 
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A probabilistic status set VS is action consistent w.r.t. O iff for every action- 
constraint of the form 

{ai{Xi),...,ak{Xk)}^X (2) 

either O ^^^'^^ x or {ai(Xi), . . . , afc(Xfc)} % Do (VS). 
The following example illustrate the concept of deontic and action consistency. 



Example 5.6 Suppose we have a resource allocation agent having two actions 
— send-A{) and send^BQ — each of which sends a unit of the resource respec- 
tively to agents A B. To execute either of them, we need to have at least one 
unit of resource, and to execute them together we need at least 2 units: 

Pre{send-A{)) = in(X, allocator: avaiLrscQ) & X > 0. 
Pre{send-B{)) — in(X, allocator: avail_rsc{)) & X > 0. 
{sendJ.o_A{),send_toJ3{)} ^^ in(X, allocator: avaiLrscQ) & X < 2 

Suppose the agent's current state O is one in which availjrsc{) returns 1. Then 

PS = {PsendJo_A{), Do send_to_A{), 
Do sendJo_BQ, OsendJo-BQ} 

is deontically consistent (there are no W and F atoms at all, and the action 
preconditions are true), but not action consistent. 

The deontic and action closure of a probabilistic status set VS is defined in 



exactly the same way (see appendix, Definition A. 3 on page 39) as in the non- 
probabilistic case. 

5.4 Probabilistic State Consistency 

The final requirement of a feasible probabilistic status set ensures that the new 
state that results after concurrently executing a set of actions is consistent with 
the integrity constraints. 

O satisfies the integrity constraint "0 => x iff either O ^^^'^' V' or O \^'-^-^' x- 

Definition 5.7 (Probabilistic State Consistency) A probabilistic status set 
PS is probabilistically state consistent w.r.t. Og iff the new state, O'g = 
conc(Do (PS), Os) obtained after concurrently executing all actions of the form 
Do a G PS satisfies all integrity constraints. 

The following example illustrates the concept of probabilistic state consistency. 

Example 5.8 Suppose we have a vehicle coordination agent that tracks vehicle 
on a road (line), and makes sure that two vehicles do not collide. Such an agent 
may have the integrity constraint 

in(X, geo : getposition{a)) & in(Y, geo : getposition{h)) ^ X ^Y 
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It may be able to perform an action move.f orward{a): 

Pre{move_f orward{a)) = in(X, geo : getposition{a.)) 
Del{move-f orward{a)) = in(X, geo : getposition{aL)) 
Add{move-f orward{a)) = in(X + 1, geo : getposition{a.)) 

In a state O where geo : getpo.sition{a.) returns 200, and geo : getpo.sition{h) 
returns 201, the status set 

VS = {Pmove-forward{a),T)omove_forward(a)} 

is not state consistent, as executing Do (VS) leads to where both agent a and 
agent b are in position 201, violating the above integrity constraint. 

5.5 Feasible Probabilistic Status Sets 

The meaning of a pap (w.r.t. a given state) may be characterized via those prob- 
abilistic status sets that satisfy the conditions of closure under program rules, 
deontic/action consistency and probabilistic state consistency. Such probabilis- 
tic status sets are said to be feasible. 

Definition 5.9 (Feasible Probabilistic Status Set) Suppose W is an agent 
program and Og is an agent state. A probabilistic status set VS is feasible for 
W on Os if the following conditions hold: 

{VS 1): App-p-p ^o g{V S) C VS (closure under the program rules); 
{VS 2): VS is deontically and action consistent (deontic/action consistency); 
{VS 3): VS is action closed and deontically closed (deontic/action closure); 
{VS 4): VS is state consistent (state consistency). 

paps may have zero, one or many feasible status sets, as seen via the following 
examples. 

Example 5.10 Consider the following agent program. 

Psend_wam(i80) ^- 
Fsend-warn{t80) ^- 

In any agent state Og such that Os ^[i^^l Pre{send-warn{t80)), the above 
program cannot have any feasible probabilistic status set VS. This is because 
closure under program rules requires that P send jwarn{t80),F send jwarn{t80) 
are both in VS, but this causes VS to violate deontic consistency. 

In contrast, consider the following one-rule program for checking the power 
level of surveillance equipment. 

Opower_warn() ^- in{X, surv: powerlevel{)) Sz X < 2000. 
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Suppose surv : powerlevelQ returns 1000 in some state Os, and suppose power .warnQ 
has no preconditions. If no integrity and action constraints are present, then 
this pap has exactly one feasible status set, viz. for power jwarn{), and without 
action 

{OpowerjLuarn() , Do power _warn{) , Ppower_warn()} 

in 05- 

Now let us consider a pap which says that one of the two agents a, b must 
be warned (if it is active). Furthermore, if b is to be warned, its regular (non 
emergency) channel must not be on. 

Fopen_ch{b) ^- -iFopen_ch{b) & T)owarn_ag{b). 
Tiowarn_ag{a) ^- in(a.^ surv: activeagentsQ) &z -iDowarri-agib). 
Owarn-ag{b) ^- [n{'b, surv: activeagentsO) &z -iT>owarn-ag{a). 

We assume the absence of integrity constraints, and preconditions for all actions. 
However, the following action constraint is present: 

{warn_ag{a),warn_ag(b)} ^^ . 

If surv: activeagents{) returns {a, 6} in state Os, then the above program has 
several feasible status sets: 

{Fopenjch{b), Owarn_ag{b) ,1)0 warn_ag{b) , Pwarn-ag{b)} 
{Fopenjch{b), Do warn_ag{a) , Pwarn-ag(a)} 
{Do war n_ag (a) , Fwarn_ag(a)} 

Notice that no feasible status set contains both Do warn_ag{a) and Do warn_ag(b) . 

5.6 Rational Probabilistic Status Sets 

As seen from the above examples, feasible status sets may contain action status 
atoms that are not required for feasibility. Rational probabilistic status sets 
refine this definition. 

Definition 5.11 (Groundedness; Rational Probabilistic Status Set) 

A probabilistic status set VS is grounded if there is no probabilistic status set 
VS' ^ VS such that VS' C VS and VS' satisfies conditions {VS1)-{VS3) of a 
feasible probabilistic status set. 

VS is rational iff it is feasible and grounded. 



Example 5.12 Consider the last case in Example 5.10, Only two of the listed 
feasible status sets are rational, viz. 

{Fopen_ch{b), Owarn_ag{b), Do warn_ag{b) , Fwarn_ag{b)} and 
{Do warn_ag{a) , Fwarn_ag{a) } 
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5.7 Reasonable Probabilistic Status Sets 

As we can see from the preceding example, certain action status atoms may be 
true in a rational status set even though there is no rule whose head contains 
(or implies) that action status atom. The concept of a reasonable status set 
(which is derived from the well known stable model semantics of logic programs 
( pelfond and Lifschitz 1988| )) prevents this. 

Definition 5.13 (Reasonable Probabilistic Status Set) Suppose VV is a 
pap, Og is an agent state, and VS is a probabilistic status set. 

1. If VV is a positive (i.e. B~^(r) = for all r G VV), then VS is a 
reasonable probabilistic status set for VV on Os, if, by definition, VS is 
a rational probabilistic status set for VV on Os- 

2. Otherwise, the reduct ofVV w.r.t. VS and Os , denoted by red^^ {VV, Os), 
is the program which is obtained from the ground instances of the rules in 
VV over Os as follows. 

(a) First, remove every rule r such that B^g{r) D VS ^ 0; 

(b) Remove all atoms in B^^{r) from the remaining rules. 

Then VS is a reasonable probabilistic status set for VV w.r.t. Os, if it 
is a reasonable probabilistic status set of the program red^^{VV, Os) with 
respect to Os. 

The following example illustrates the concept of a reasonable status set. 



Example 5.14 Consider again the last case in Example 5.1C. Only one of the 
listed feasible status sets is reasonable, viz. 

VS = {Do warn_ag{a) ,Pwarn_ag{a)} 

To see why this probabilistic status set is feasible, note that the reduct of VV 
w.r.t. VS is: 

T>owarn_ag{a) <— in{&, surv: activeagents (J). 

whose (unique) rational status set is obviously VS. 

5.8 Semantical Properties 

In this section, we prove some properties about the different semantics described 
above. 

Proposition 5.15 (Properties of Feasible Status Sets) Let VS be a fea- 
sible probabilistic status set. Then, 

1. If Do (a) e VS, then Os h'^'^' Pre{a); 
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2. If Pa ^ rS, then Do (a) ^ VS; 

3. IfOa e VS, then Os h'^'^' Pre{a); 
I IfOa e VS, then Fa ^ VS. 

The following theorem says that reasonable status sets are rational. 

Theorem 5.16 (Reasonable Status Sets are Rational) LetVV be a prob- 
abilistic agent program and Og an agent state. Every reasonable probabilistic 
status set of VV on Os is a rational probabilistic status set of VV on Os- 

Given any pap VV and agent state C5, we may define an operator that maps 
probabilistic status sets to probabilistic status sets as follows. We use then 
notation D-C1(P5) to denote the closure of VS under the rule Oa G VS =4> 
Pa e VS and A-C\{VS) to denote the closure of VS under the rules Oa G 
VS^'Doa^VS and Do a e 7^5 ^ Oa G VS. 

Definition 5.17 {T-p-p^Q^ Operator) SupposeVV is a probabilistic agent pro- 
gram and Os is an agent state. Then, for any probabilistic status set VS, 

Tpp^OsCPS) = Apppp^os{'PS)li'D-Cl{VS)UA-Cl{VS). 

Note that as D-C1(7'S') C A-Cl{VS), we may equivalently write this as 

Trr^OsCPS) - Appvv^Os(T^S)U A-Cl{VS). 

The following property of feasible probabilistic status sets is easily seen. 

Lemma 5.18 (VS as Preflxpoint of T-p-p^o^) LetVV be a probabilistic agent 
program, Os be any agent state, and VS be any probabilistic status set. If VS 
satisfies conditions (VSl) and {VS3) of feasibility, then VS is pre-fixpoint of 
Tvr,Osj *•£•; Tpp^osCPS) C VS. 

Proof: Suppose Op(a) e Tpp^OsCPS) ^ Apppp^OsO^S) U A-Cl{VS). 
Then we have either Op(a) e Apppp^OsCPS) or Op(a) e A-C1(7'S'). By 
condition {VS 1) defining a feasible probabilistic status set, we know that 
Apppp ^OsCPS) C VS. By condition (VS 3), VS ^ A-C\{VS) and hence, 
A-Cl{VS) C VS. Therefore, Tpp^OsCPS) C VS. | 

The following theorem says that in the absence of integrity constraints, a 
pap has a rational probabilistic status set if and only if it has a feasible one. 

Theorem 5.19 (Existence of Rational Probabilistic Status Sets) LetVV 
be a probabilistic agent program. IfTC — 0, then VV has a rational probabilistic 
status set if and only if VV has a feasible probabilistic status set. 
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6 Computing Probabilistic Status Sets of Posi- 
tive paps 

In this section, we present a sound and complete algorithm to compute the 
unique reasonable status set of a positive pap. For this purpose, we use a variant 
of the T-p-p^o^ operator introduced earlier. This operator, denoted S■p■p^Os^ is 
defined as 

Spp^OsCPS) = A-Cl(Apppp^Os(^^))- 
Computationally, we may compute the operator S-p-p.Os using algorithm p.l] 
below. 

Algorithm 6.1 {S-p-po^ Computation for Positive paps) 

Compute-Sppos (^s- agent state, PS: probabilistic status set) 

(-k the probabilistic agent program VP is positive; *) 

(-k Input: an agent state Og, and a prob. status set VS -k) 

(k Output: a deontically and action consistent set Spp,Os{'PS) (if existent) -k) 

(k or "no consistent set exists" k) 

1. X := VS; 

2. for each rule r E W 

3. for each ground instance rO of r 
4- if rO ~ Op a <— F, Li, . . . , L„ and 

O5 1= F and {Li, . . . , L„} C VS and 

for every atom Op'{l3) e {Li, . . . , L„} U {Op{a.)} 

such that Op' e {P,0,Do}; O5 h'^'^' Pre{[3) 

5. then X := X \J A-Cl({Opa}), 

6. if X contains (Oa and 'Wa) or (Pa and Ya) 

7. then Return "no consistent set exists" . 

8. Return X . 
end. 



The behavior of Algorithm 6.1 is illustrated by the following example. 



Example 6.1 Consider the following program, saying that whenever a (prob- 
ably) enemy vehicle Y is detected, a warning message about Y is sent to a 
friendly source and the agent perfroming the detection is not allowed to move. 

Osend-warn(Y) ^- in(F, surv :/iZe(iinagedb)) & 

in(Y, suTv :rv identify {Y)){[Q.h, 1.0], ®ig) k 
O sendjwarn {X). 
FmoveQ ^~ Tiosendjjjarn{X). 

Osendjwarn{X) ^- in(F, surv :/i^e(iinagedb)) & 
in(X, surv :rv identify (F)) & 
in(X, surv : enemy vehicles {))) : ([0.5, 1.0], (X)ig). 
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Moreover, assume that in the current state C5, surv:/iZe(imagedb) returns 
imagel, surv : identify (imagel) returns the random variables ({i80}, {(t80, 0.6)}) 
and ({t72}, {(i72,0.5)}), and surv: enemy vehicles {) returns i80. 

Now we apply Algorithm |6.1| to compute S-p-p^Osi^s, 9)- Step 1 sets X = 0, 
step 2 selects the first rule, while step 3 considers all ground instances (of the first 
rule) whose body's truth is checked in step 4: no instance satisfies it (because 
VS is empty), so nothing happens. The same result is obtained when step 2 
considers the second rule. Eventually, step 2 considers the third rule, which 
satisfies the condition of step 4 with its head instantiated to Osendjwarn{t80). 
Step 5 inserts Osendjwarn(t80), Do sendjwarn{t80) , Fsendjwarn{t80) into X. 
There is no deontic inconsistency, so the check n step 6 fails and then we jump 
to step 8, which returns the result: 

X — { Osendjwarn(t80),T>o send_warn{t80),Fsend-warn{t80) }. 

The operator S-p-p q^ may be iteratively applied as follows. 






00 

4=0 

The following theorem says that for positive paps, operator Spp^Os is mono- 
tonic, continuous, and has a (unique) least fixpoint. 

Lemma 6.2 (Monotonicity and Continuity of Spp^Os) Suppose W is a 
positive pap. Then the operator Spp^Og is monotone and continuous, i.e. 

1. rSi c VS2 => Spp^Osi'PSi) c Spp,Osi'PS2), 

2. Spp^OsiWiLoT^Sa) = [J'^qSpp,Os{T^S^) for any chain PSq C VSi C 
VS2 C • • • of probabilistic status sets, 

3. S^-p Q is a fixpoint of Spp_Os ■ Moreover, it is the least fixpoint of 
Spp^Os ■ 

Proof: By the well known Knaster/Tarski theorem, 3. follows from 1. and 2.. 
To show 1., let VSi C 1^82- But then 

Apppp o^CPS"!) C Apppp o^(7'S'2), 

because of the monotonicity of App-pp 0^0 (see ( [Lloyd 1987 ; Apt 1990 )). This 



implies A-Cl(App.ppo^(7'S'i)) C A-Cl(Apppp ^^(T'S'a)). 

2. follows similarly from the continuity of Apppp q^ () and the fact that 

A-C1((J Appp^^o^(7'^,)) = [JA-C\{A^Y>PP,Os('PS,)). 

i=0 i=0 
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The following example shows the computation of Spp q^ . 



Example 6.3 Consider the program in Example |6.l| . Applying the operator 
^vv.Os iteratively, we obtain: 

^vp Os ^ {Osend_warn{t80),T)o send_warn{t80),Psend-warn{t80)} 
^vv.Os ^ S]:,'p Q^ U {Osend_warn{t72),'Dosendjwarn{t72), 
Psend_warn(t72), FmoveQ} 

S3 c2 Q^ 



The following results tell us that Lemma 5.18| , which holds for arbitrary 



programs, can be strengthened to the case of positive probabilistic programs. 

Theorem 6.4 (Rational Probabilistic Status Sets as Least Fixpoints) 

Suppose VP is a positive pap, and Os is an agent state. Then: VS is a ratio- 
nal probabilistic status set if and only ifPS = lfp{S-p-p^Os) o-ndVS is a feasible 
probabilistic status set. Recall that Ifp stands for least fixpoint. 



Corollary 6.5 Let VP be a positive probabilistic agent program. Then, on 
every agent state Os, the rational probabilistic status set of W (if one exists) 
is unigue, i.e., ifPSjVS are rational probabilistic status sets for W on Os, 
thenVS = VS'. 

An important corollary of this theorem is that to compute a reasonable feasible 
status set of a pap, all we need to do it to compute lfp{S-p-p,Os)- This may be 
done via Algorithm Compute- //p below. 

Algorithm 6.2 (Reas. Prob. Status Set Computation for Positive paps) 

Compute-lfp (T-p-p^Og): agent state, W: probabilistic agent program) 

(-k the probabilistic agent program VP is positive; -k) 
(-k Input: an agent state Os, and a pap W -k) 

(-k Output: a reasonable probabilistic status set k) 



change .-— true; X :— 0; 
while change do 

newX = Compute-S-p-pos (^)> 
if new X := "no consistent set exists" 
then return no reasonable prob. status set exists. 
if X ^ newX then X := newX 
else change ;= false. 
end while; 
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9. if X satisfies all the following conditions 

10. • DoaeX^O h'^'^' -P?-e(a); 

11. • The new state obtained by executing conc({Doa | Do a G X\ 

12. satisfies the integrity constraints; 

13. • {Do a I Do a £ X} satisfies the action constraints. 

14. then return X 

15. else return no reasonable prob. status set exists. 
end. 



Theorem 6.6 (Polynomial Data Complexity) Algorithm Compute-lfp has 
polynomial data- complexity. 

Proof: It is easy to see that the while loop of the algorithm can be executed in 
polynomial time (data-complexity). Checking if X satisfies the three conditions 
at the end of the algorithm are each polynomial time checks (assuming the 
existence of a polynomial oracle to compute code call conditions). | 

The following example walks the reader through the detailed working of this 
algorithm on the motivating example pap introduced earlier on in this paper. 

Example 6.7 We apply the above algorithm to the program (and agent state) 



of Example 6.1 



Step 1 initialize X to 0, i.e. to S^p q , while steps 2-8 iteratively apply 
the procedure which implements the operator S-p-p_Os ■ 

At the first iteration, in step 3 newX becomes Spp q (shown in Exam- 



ple 3.3); since there are no deontic inconsistencies, the test in step 4 fails, 
and then we skip to step 6 which will assign X := newX, and then the 
cycle starts again. 

• At the second iteration newX becomes S|,p q^ , then it is still inconsistency- 
free and different from X, so that we go on for another iteration. 

• The third iteration is also the last one, since S|,p ^ = S^^ ^ , and so 
we skip to the tests in steps 10-13. 

• In our example, the preconditions of all actions are empty and then sat- 
isfied, there are not integrity constraint and then X is trivially integrity 
consistant, and eventually, there are no action constraints and then X 
is also action consistent. X is then returned as the (unique) reasonable 
status set of the program. 

6.1 Agent Programs are Probabilistic Agent Programs 

In this section, we show that the concept of an agent program defined by 



Eitcr, Subrahmanian, and Pick (1999) is a special case of the framework de- 



fined here. Hence, paps generalize the Eiter et. al. semantics. Furthermore, 
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algorithm Compute- Z/p may be used to compute reasonable status sets of pos- 
itive agent programs. First, we show how agent programs may be captured as 
paps. 

Definition 6.8 Let W be a probabilistic agent program, VS a probabilistic 
status set and O a probabilistic agent state. Assume further that each random 
variable contains exactly one object with probability 1. Then we can define the 
following mappings: 

Redi(-), which maps every probabilistic code call of the form ({o}, 1) to a: 

i?erfi(({oRv},l))=o. 

Red2(-), which maps annotated code call conditions to code call conditions by 
simply removing the annotations and the conjunction strategy: 

Red2{x ■■ ([aii,ai2],(8))) = X- 

We can easily extend Red2{-) to a mapping from arbitrary conjunctions of 
annotated code calls to conjunctions of code calls. 

Red3(-), which maps every probabilistic agent program to a non-probabilistic 
agent program: it clearly suffices to define Red^{-) on probabilistic agent 
rules. This is done as follows 

RedziA <— r, Li, . . . , L„) = A^ Red2(T),Li, . . . , L„. 
Under the above assumptions, the following theorem holds. 

Theorem 6.9 (Semantics of Agent Programs as an Instance of paps) 

Suppose all random variables have the form 

{{objectBv},!). 

Then: (x '. ([aii, ai2], (X") is a ground annotated code call condition, Os an agent 
state) 

Satisfaction: the satisfaction relations coincide, i.e. 

0^[aii,ab]^. ([ai^^ais],®) if and only if O ^ Red2{x ■■ ([aii, aia], ®)). 

App-Operators: the App-Operators coincide, i.e. 

ApPRed3{VV),Osi'PS) ^ App-pv.OsCPS), 

where the operator on the left hand side is the one introduced in Defini- 



tion 



A. 4 on page [39, 
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Feasibility: Feasible probabilistic status sets coincide with feasible status sets 
under our reductions, i.e. VS is a feasible probabilistic status set w.r.t. 
VP if and only ifVS is a feasible status set w.r.t. Red'i{W) . 

Rational: Rational probabilistic status sets coincide with rational status sets 
under our reductions, i.e. VS is a rational probabilistic status set w.r.t. 
VP if and only ifPS is a rational status set w.r.t. Red^iVV). 

Reasonable: Reasonable probabilistic status sets coincide with reasonable sta- 
tus sets under our reductions, i.e. VS is a reasonable probabilistic status 
set w.r.t. VV if and only ifVS is a reasonable status set w.r.t. Red'i{VV) . 

Computation of Status Sets: The computations of probabilistic status sets 



given in Algorithms 6.1 on page [2q and \6.i\ on page \2^ for a pap VV 



reduce to the computation of status sets for Red^lVV). 

Proof: The first two statements are immediate. Feasibility requires checking 
conditions {VS1)-{VS4:), and therefore reduces to the first two statements. Ra- 
tional and reasonable status sets are handled in a completely analogous manner. 
That our algorithms reduce to the non-probabilistic case under our general 
assumption is trivial: the difference is only the satisfaction relation ^[^'^l which, 
by the first statement, coincides with |=. | 



7 Probabilistic Agent Programs: Kripke Seman- 
tics 

The definition of a feasible status set given in Section S makes several simpli 



fying assumptions. First (see Definition 5.3), it assumes that an action can be 
executed only if its precondition is believed by th e ag ent to be true in the agent 
state with probability 1. Second (see Definition p.5| ), every action that is per- 
mitted must also have a precondition that is believed to be true with probability 
1. In this section, we propose a Kripke-style semantics for agent programs that 
removes these conditions. 

To do this, we will start by noting that in a probabilistic state C, the agent 
returns a set of random variables for each code call. Every probabilistic state 
implicitly determines a set of (ordinary) states that are "compatible" with it. 
We use the notation eval(Q : /(di, . . . , d^), O) to denote the result of evaluating 
the code call a:/(di, . . . , dn) w.r.t. the state O. 

Definition 7.1 (Compatibility of State v^r.r.t. a Probabilistic State) Let 

O^ be a probabilistic agent state. An (ordinary) agent state O is said to be com- 
patible with O^ iff for every ground code call a : /(di, . . . , d^), it is the case that 
for every object o G eval(Q :/(di, . . . ,dii),C'), there exists a random variable 
{X, p) e eval(Q :/(di, . . . , dn), O^) such that o G X and p{o) > 0, and there is 
no other object a' G X such that a' G eval(a :/(di, . . . , dn), O). 
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The following example illustrates this concept. 

Example 7.2 Consider a probabilistic agent state O^ with only two code calls 
surv: identify (±m&gel) and surv : location(±magel), which respectively return 
the random variables 

({i80, t72, t70}, {(t80, 0.3), (i72, 0.7), (t70, 0.0)}) 

and {{Loc2}, {{Loc2, 0.8)}). The agent states compatible w.r.t. O^ are described 
in the following table: 



State 



Vehicle Location 



State 



1 


none 


none 


2 


t80 


none 


3 


t72 


none 



Vehicle Location 



4 


none 


Loc2 


5 


t80 


Loc2 


6 


tl2 


Loc2 



The object "t70" in the first random variable has a null probability, and hence 
it does not appear in any compatible agent state. In states 1-3, the location is 
unknown. In states 1 and 4, the vehicle in the image is unknown. 

We use the notation COS(C''') to denote the set of all ordinary agent states 
that are compatible with a O^. We now define the notion of a probabilistic 
Kripke structure. 

Definition 7.3 (Probabilistic Kripke Structure) 

A probabilistic Kripke structure is a pair {S, p) where S is a set of ordinary 
states, and p : S ^ [0,1] is a mapping such that J2o€S P(^) ~ ^■ 



Definition 7.4 (Compatible Probabilistic Kripke Structure) Let O^ be 

a probabilistic agent state. A coherent probabilistic Kripke structure (€05(0^), p) 
is said to be compatible with O^ iff for every ground code call Q:/(di, . . . ,dn), 
for every random variable {X,p') G eval(Q :/(di, . . . ,dn),C"'), and for each 
object 0, it is the case that: 



E 



p{o) 



oGeval(a : /(di,... .dn 



P'(o) 




otherwise 



By definition, two distinct objects from the same random variable cannot ap- 
pear in the same compatible state. If such a random variable has a complete 
probability distribution, then in any compatible Kripke structure, the sum of 
the probabilities of the states containing one of its objects is equal to 1. This 
means that any (compatible) agent state containing no such objects will have a 
null probability, avoiding the intuitive inconsistency pointed in example 7.2. 



Example 7.5 Considering the situation in example 7.1 on the previous page. 



a probabilistic Kripke structure compatible with O^ is (COS(C'^), p), with the 
following probability distribution: 
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state 



Probability State 



4 

0.1 5 

0.1 6 



Probability 





0.2 
0.6 



The following result says that compatible Kripke structures always exist. 

Theorem 7.6 (Existence of Compatible Probabilistic Kripke Structure) 

Suppose OP is a probabilistic agent state. Then there is at least one probabilistic 
Kripke structure which is compatible with it. 

Hence, given a probabilistic agent state O^, we use the notation PKS(C'p) to 
denote the set of all probabilistic Kripke structures compatible with O^ — this 
set is guaranteed to be nonempty by the preceding result. However, in almost 
all cases, PKS(C'p) contains an infinite number of elements. 

Theorem 7.7 (Existence of Infinitely Many Kripke Structures) If a prob- 
abilistic state O^ contains at least two random variables (returned by the same 
code call or by two distinct ones) containing at least one object with associated 
probability < p < 1, then there exist an infinite number of probabilistic Kripke 
structures compatible with O^ . 

We are now in a position to specify what it means to execute an action in a 
probabilistic Kripke structure. 

Definition 7.8 (Action Execution) Suppose O^ is a probabilistic agent state. 
A ground action a is said to be possibly executable in O^ iff there is at least one 
probabilistic Kripke structure (COS(C"'), p) G PKS(C'^), and an ordinary agent 
state O in COS(C"') in which the precondition of a is true such that p{0) > 0. 
In this case, O witnesses the executability of a. 

We say that a is executable with probability p in O^ if by definition, 

p = min{p{0) I O G COS(C''') witnesses the executability of a}. 

The following example illustrates this definition. 



Example 7.9 Let us consider the probabilistic agent state of examples l.i and 1. 
and the following actions: 



ai 
as 



Pre{ai) = in(t70, surv : identify (ima^gel)) 
Pre{a2) = in(X, surv : location {image 1)) & X ^ Loc2 
Pre{a-i) = in(Loc2, surv : ZocoiJon( image 1)) & 
in(t80, surv : identify (im&gel)) 



As stated before, the object "t70" cannot appear in any compatible agent state, 
and hence the action ai is not possibly executable. On the other hand, the 
precondition of ot2 requires the existence of an object other than "Loc2", which 
is known to be the only one possibly returned by the corresponding code call. 
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and hence 02 is not possibly executable either. Eventually, the precondition 
of as requires the presence of both objects "Loc2" and "t80", which is true in 
the agent state number 5 described in the above examples. As this state has a 
non null probability, a^ is possibly executable and the agent state witnesses its 
executability. 

We are now ready to define the new probabilistic Kripke structure that 
results when an action is executed in it. 

Definition 7.10 (Result of Action (6', 7)-Execution) Suppose O^ is a prob- 
abilistic agent state, a{X) is an action and j is a ground substitution for all 
variables occurring in the precondition, add, and delete list of a{X)9. Let 
(COS(C),p) be a probabilistic Kripke structure that contains a witness O to 
the possible executability of a{X)9. The result of executing action a{X) under 
substitutions 9,j in probabilistic Kripke structure PKS{0^) — (5, p) is a new 
Kripke structure (S' , p') defined in the following way: 

1. S' = {niap,-^^ g ^(^) I £ i^} where map is defined as follows: 

^„„ . (.^. ( apply{a{X),e,j,0) if O e W 
^^P»(x).e.^^) - I otherwise 

2. p' is defined as follows: 

p'iO') = J2{piO) I O' = map^^^yg^^iO)} 

In the above definitions, W is the set of all witnesses in S to the executability 
ofa{X)0j. 

The result of executing action a{X) under substitutions 9,^ in a set /C of 
probabilistic Kripke structures is {S' \ S' is obtained by executing a{X) under 
substitutions 6,"f on S £ JC}. 

The definition causes the agent states in which a's precondition is true to change, 
while those in which a's precondition is false stay unchanged. The probability of 
each final state is the sum of the probabilities of the corresponding (old) states. 
This is illustrated by the following example. 

Example 7.11 Let us consider the compatible probabilistic Kripke structure in 



Example l.l, and the action eraseiX): 

Pre: in(X, surv : identify {imaigel)) 

Del: in(X, surv : identify {im.a.gel)) 

Add: 

The result of executing action erase{X) under substitutions {X/t80},e, is the 
probabilistic Kripke structure {S',p'), briefly described by the following table: 
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state 


Vehicle 


Location 


Probability 


a 


none 


none 


0.1 


b 


t72 


none 


0.1 


c 


none 


Loc2 


0.2 


d 


t72 


Loc2 


0.6 



i.e., the states 1 and 2 merge together yielding the new state "a" and their 
probabilities are summed. Similarly, states 4 ci^d 5 yield the new state "c". 

The following result states that our definitions are coherent. 

Proposition 7.12 (Closure of Probabilistic Kripke Structures) The re- 
sult of (0,^)- execution of an action in a probabilistic Kripke structure is also a 
probabilistic Kripke structure. 

Proof: Let {S, p) be the original Kripke structure, and (5', p') the result of 
executing the action. We just need to show that ^{p'{0') \ O' £ S'} = 1. Using 
Definition 7.10| on the preceding page: 



^ p'(o') =Y. Y. p(^) = E p(^) = 1 

0'£S' O'eS' 0'=map(0) oes 



8 p-Feasible Status Sets 

Probabilistic Feasible Status Sets prevent an action from being executed unless 
its precondition is known for sure to be true (which is exactly the intuitive 
reading of O^ ^[^'^1 Pre{a), for a probabilistic agent state O^ and an action 
a). Analogously, an action constraint has to be checked only if its precondition 
is certainly true. Finally, state consistency requires that the execution of the 
actions does not corrupt the consistency of the original agent state, i.e. it has to 
lead to an agent state where the integrity constraints are with 100% probability. 
In this section, we define the concept of p- feasibility, where p is a probability, 
p-feasibility weakens the above requirements to only requiring that preconditions 
are true with probability p (or higher). 

Definition 8.1 (Operator p-App-p-p^opi'PS)) Suppose VP is a probabilistic 
agent program, O^ is a probabilistic agent state, and VS is a probabilistic status 
set. Then 

p-App-p-pQp{VS) = {Opa\ Op a is the head of a ground instance of a rule 

r in W and: 

1. B+{r) C VS and ^.B-,{r) DVS ^ 0; 

2. For every annotated code call condition x '■ ([aii, ai2], (X") in the body of r, 
it is the case that O^ |=[=''i'=''2l ^ : ([aii, ai2], (g)); 
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3. if Op G {P, 0,Do}, then the preconditions of a are true with probability 
p or greater, i.e. O^ '^a>>'^\ Pre{a); 

4-. for every action status atom of the form Op (3 in B'^g{r) such that Op € 
{P,0,Do}, the preconditions of (3 are true with probability p or greater, 
i.e. 0P [==[P^il Pre{(i) } 

The only difference between this definition and that of Apppp^o^ {T^S) is that 
the entailment |=[i'il is replaced by the more general ^I^'^l. 

Definition 8.2 (Deontic and Action p- Consistency) A probabilistic status 
set VS is deontically p-consistent with respect to a probabilistic agent state O^ 
if, by definition, it satisfies the following rules for any ground action a: 

• IfOae VS, then Wa ^ VS. 

• IfPae VS, then Fa ^ VS. 

• // Pa G VS, then the preconditions of a are true with probability p or 
greater, i.e. QP h'^'^' Pre{a). 

A probabilistic status set VS is action p-consistent with respect to an agent state 
C' iff for every action constraint of the form 

{ai{Xi),...,ak{Xk)}^X (3) 

either O^ ^[^^il x or {a^{X^), ... , ak{Xk)] % VS. 

Generalizing probabilistic state consistency to p-probabilistic state consistency 
may be done in two ways. 

Definition 8.3 (Probabilistic State p-Consistency) A probabilistic status 
set VS is weakly probabilistically state p-consistent w.r.t. state O^ iff the new 
state, QP ~ conc(Do {V S) , O^) obtained after concurrently executing all ac- 
tions of the form Do a G VS satisfies all integrity constraints with probability 
greater than or equal to p, i.e. for every integrity constraint ip ^ X either 
Qp' ^[p4] ^ or QP' h'^'^' X- 

We say that a probabilistic status set VS is strongly probabilistically state p- 
consistcnt w.r.t. state OP iff the new .state OP satisfies the following condition: 
if OP satisfies the integrity constraints with probability > q (q € [0, l]j then also 
OP does so. 

These definitions induce two types of feasibility for arbitrary probabilities p. 

Definition 8.4 (Weak (resp. Strong) p-Feasibility) Let VV be an agent 
program and let OP be an agent state. Then, a probabilistic status set VS is 
a p-feasible probabilistic status set for VV on OP , if the following conditions 
hold: 
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{p-VS 1): p-App-p-p^QplVS) C VS (closure under the program rules); 

{p-VS 2): VS is deontically and action p-consistent (deontic and action p- 
consistency); 

{jp-VS 3): VS is action closed and deontically closed (deontic and action clo- 
sure); 

(jp-VS 4): VS is weakly (resp. strong) state p-consistent (state p- consistency). 



Remark 8.1 If S* is a p- feasible probabilistic status set for VV on O^ and 
< <Z < P, then S is not always q-feasible. Indeed, [q,l] 3 [Pj1]i ^nd then 
for any formula O^ [=[P'^1 implies that O^ \=^i'^^ (p (and analogously for 
^[p.i] and ^[''^1). This means that all preconditions of actions, preconditions 
of action constraints and integrity constraints which are verified for p are also 
verified for q. 

The problem is that p-A.pp-p-p_Qv{VS) is anti-monotonic w.r.t. p, as a smaller 
value for p may allow a larger set of rules to be Arable. Then the closure under 
the program rules is not guaranteed any more. 

The following example illustrates this point. 

Example 8.5 Consider the following trivial program: 

Do a ^- . 

where Pre{a) = m(a,d :/()), and eval(d:/(), O^) = {({a}, (0.7))}. Suppose 
VS ~ 0. Conditions p-VS 2-4 are true for any value of p. Note that 0.8- 
App-p-p^Osi'P^) = nd hence, VS is 0.8-feasible. In contrast, we see that 
0.6-Appp-p^C)<,(PS') — {Do a} % VS, and hence VS is not 0.6-feasible. 

We can easily see that probabilistic feasibility is a particular case p-feasibility: 



Proposition 8.6 Let VV he a probabilistic agent program and let O^ be a con- 
sistent (or equivalently 1-consistent) agent state. Then, a probabilistic status 
set VS is a feasible probabilistic status set if and only if it is weakly 1- feasible 
if and only if it is strongly 1- feasible. 

Proof: All definitions for weak p-feasibility trivially coincide with those for 
feasibility if p=l. The distinction between weak and strong p-feasibility is just in 
the definition of p-consistcncy, and we can easily see that for p=l they coincide, 
since a probability greater or equal to 1 cannot be but equal to 1. | 
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8.1 p-Rational and p-Reasonable Status Sets 

The notions of Rational and Reasonable Status Sets can be straightforwardly 
extended to those of p-Rational and p-Reasonable status sets. 

Definition 8.7 (p-Rational Status Set) A probabilistic status set PS is a 
p-rational probabilistic status set, if VS is a p-feasible probabilistic status set 
and there exists no probabilistic status set VS' C VS satisfying conditions {p — 
VSl)-{p — VS3) of a p-feasible probabilistic status set. 

Obviously, in the case that TC = (i.e., there are no integrity constraints) 
p-rational status sets are simply inclusion-minimal feasible status sets. 

Definition 8.8 (p-Reasonable Probabilistic Status Set) LetVV be a prob- 
abilistic agent program, let Os be an agent state, and let VS be a probabilistic 
status set. 

1. IfW is a positive probabilistic agent program, then VS is a p-reasonable 
probabilistic status set for VV on Os, if, by definition, VS is a p-rational 
probabilistic status set for VV on Os . 

2. Exploiting the definition ofred^{VV, Os) (see Definition \5.13l on page\ldi), 
VS is a p-reasonable probabilistic status set for VV w.r.t. Os, if it is a 
p-reasonable probabilistic status set of the program red^^{VV,Os) with 
respect to Os. 

It is easy to verify that all p-reasonable probabilistic status sets are p-rational 
probabihstic status sets: 

Proposition 8.9 (p-Reasonable Status Sets are p-Rational) Let VV be 

a probabilistic agent program and Os an agent state. Then, every p-reasonable 
probabilistic status set of VV on Os is a p-rational probabilistic status set of 
VV onOs- 



Proof: Identical to the proof of Proposition 5.16 on page 03. | 

As in Section on page EO, we can define a fixpoint operator and build an 
algorithm on its top to compute p-reasonable status sets for positive programs. 

Definition 8.10 (Operator p— Spp o^) 

^-Svv.Osi'PS) = A-Cl(p-Apppp^Os(^^))' 
Op era tor p— Spp Os *^^n. be computed by an algorithm identical to Algo- 



rithm xl on page |2y, but for step 4, where the entailment ^[i^^l has to be 
replaced by ^[^'^1. p— S-pp^o^ is monotonic and continuous and has a unique 
least fixpoint. 
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Lemma 8.11 (Monotonicity and Continuity of p—S-p-p^Os) Suppose W is 
a positive pap. Then the operator p—S-p-p^Os *■' monotone and continuous, i.e. 

1. VSi CVS2^P- Spp^osCPSi) c p - Spp,Os {VS2), 

2. p—Sp-p Q is a fixpoint of p—Spp^Og. Moreover, it is the least fixpoint 
of p—Spp_Og. (We assume the iterations of p—Spp^Os '^'^^ defined in the 
same way as the iterations of Spp_Os)- 

The following result now follows immediately and has a proof similar to that of 



Theorem 5.4 on page 



Theorem 8.12 (p-Rational Probab. Status Sets as Least Fixpoints) 

Let VP he a positive probabilistic agent program, and let Os be an agent state. 
Then, VS is a p-rational probabilistic status set of W on Os, if and only if 
VS = lfp{p~Spp^Os) ^'^d VS is a p-feasible probabilistic status set. 

Uniqueness of the p-reasonable status set (if it exists) holds too, and then we 
can compute it by Algorithm |6 . 2| on pagcE3, replacing-as usual — the entailment 
h'l'^l with h'^'^l- 

Unfortunately, the resulting algorithm is not polynomial because of the in- 
tegrity constraint check in steps (11)-(12). This will be discussed in detail 



in Section 8.2 below. However, when no integrity constraints are present, the 



algorithm is still polynomial. 

Theorem 8.13 (Polynomial Data Complexity) The problem of computing 
p-reasonable probabilistic status sets of positive paps without Integrity Con- 
straints (i.e., IC = %) has polynomial data- complexity. 

8.2 Checking p-Consistency of Integrity Constraints 

In this section, we provide an algorithm to check p-consistency of an integrity 
constraint IC after an action have been executed in state O^, leading to a new 
state O^' assuming that all integrity constraints are true in the original state 
O^. Suppose Oi, . . . , On are all states compatible with O^ while O'l, . . . , O^/ 
are those compatible with the new state O^ . Let Pi,p'i denote the probabilities 
of Oi, 0[ respectively. Then consider the following system of constraints: 



minimize l^o'\=icPi such that: 
(K) 'E^iP» = 1 

(CK) Vi e {1, . . . , E}. E,:0b,.60, Pi = PiObjd 

(IC) yiCeIC.l>j:oo^icP^>P 

(K ^ K') Vz G {1, . . . , N'}.pr = Y.orO,^o' P^ 
(IG) Vz e {1, . . . ,iV}.max{0,Eofcleo.V(O6jfc) + 1 - |0,|} < 

<Pi< minobjkeoApiObjk)} 
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The objective function captures the probabihty of IC being true in the new 
Kripke structure. (K) and (CK) define any arbitrary compatible Kripke struc- 
ture over N states w.r.t. O^ (which cointains E objects), (IC) expresses the fact 
that our actual state has to be p-consistent, while (K— t- K') defines the Kripke 
structure obtained after the execution of action a. Eventually, (IG) gives an 
upper and a lower bound to the probability of worlds (it extends the Bool ex- 
pression for conjunction of events of unknown inter-relation). It is easy to see 
that a straightforward implementation of this algorithm requires exponential 
time and space. 

9 Related Work 



There has been an incredible amount of work on uncertainty in knowledge based 
and database systems (Shafer and Peal 1990). However, almost all this work 



assumes that we are reasoning with logic or with Bayesian nets (KoUcr 1998) 
and most work proceeds under strong assumptions about the relationships be- 
tween events (e.g. most Bayesian approaches assume conditional independence 
between events, while other approaches such as (Fagin, Halpcrn, and Megiddo 



1990; Ng and Subrahmanian 1993b) assume that we have no knowledge of the 



dependencies between events). 

This paper introduces techniques to allow an agent developer to encode dif- 
ferent assumptions about the relationships between events, when writing prob- 
abilistic agent programs. The idea of conjunction strategies to facilitate this 
was first introduced in the ProbView system ( p^^akshmanan, Leone, Ross, and| 
3ubrahmanian 1997) in an attempt to allow users querying probabilistic rela- 
tional databases to express in their query, their knowledge of the dependencies 



between events. Later, (Dekhtyar and Subrahmanian 1997) extended the use 
of conjunction and disjunction strategies to the case of logic programs. In this 
paper, the idea of conjunction strategies are applied in the context of deontic- 
logic based agent programs. We are not aware of any extant work on allowing 

flexible dependency assumptions in t he context of logics and actions . 

Research on epistemic logic (e.g., ( Morgenstern 198^ ; Moore 1985 ; Kraus anq 



Lehmann 198S)) enables reasoning about what is known and is not known at 



a given time. However, epistemic logics have not been used as a representation 
in decision making and in automated planning systems, perhaps, because the 
richness of these languages makes efficient reasoning very difficult. In contrast, 
our framework has polynomial data complexity. 

Halpern and 'luttlc (1992) study the semantics of reasoning about dis- 
tributed systems when uncertainty is present. They develop a logic where a 
process has knowledge about the probability of events which facilitates decision- 
making by the process. We, on the other hand, consider probabilistic states, 
and as argued in ( Dix, Subrahmanian, and Pick 2000| ) this also allows us to 
reason about probabilistic beliefs, i.e. probabilities are assigned to the agents' 
beliefs about events, rather than to the events themselves. That is, in Halpern's 



work (Halpern and Tuttle 1992), the befiefs of the agent are CERTAIN, but in 



34 



our framework, the beliefs of the agent may themselves be uncertain (with the 
phenomenon when they are certain being a special case of our framework) . 

Fooic (iyyy) presented a framework that allows a natural specification of 
multi-agent decision problems. It extends logic with a new way to handle and 
think about non-determinism and uncertainty in terms of independent choices 
made by various agents, including nature and a logic program that gives the 
consequence of choices. It has general independence assumption. This work is 
more expressive than ours, but its generality leads to complexity problems and 
to difficulties in using the framework. 

Haddawy (lOill) developed a logic that allows to write sentences that de- 
scribe uncertainty in the state of the world, uncertainty of action effects, com- 
bine possibility and chance, distinguish between truth and chance and express 
information about probability distributions. He uses model theoretic semantics 
and demonstrates how his logic can be used to specification various reasoning 
and planning problems. The main purpose of the specification is to prove cor- 
rectness, and not for programming of agents. Kushmerick, Hanks, and Welcj 



|(1995D model uncertainty about the true state of the world with a probability 
distribution over the state space. Actions have uncertain effects, and each of 
these effects is also modeled with a probability distribution. They seek plans 
whose probability of success exceeds the threshold. They describe BURIDAN, 
an implemented algorithm for probabilistic planning. In contrast, we focus on 
programming agents, rather than on how agents will construct plans. Other 
researchers extended Kushmerick et al.'s model to increase the efficiency of the 



planning (Haddawy, Doan, and Goodwin 1996) or to more realistic domains 
(Doan 1996). Thiebaux, Hertzberg, Shoaff, and Schneider (1995|) developed a 



framework for anytime generation of plans under incomplete and ambiguous 
knowledge and actions with alternative and context dependent effects. 

Kaelbling, Liftman, and Cassandra (1998) propose using partially observable 
Markov decisionprocesses (POMDPs) for planning under uncertainty. Similar 
to BURIDAN they use a probability distributions over states to express uncer- 
tainty about the situation of the agent. They also consider the problem of 
non-deterministic actions and getting feedback from the environment which we 
mentioned only briefly. 

10 Conclusions 

Agents are programs that autonomously react to changes in their environment 
by taking appropriate actions. In (Eiter, Subrahmanian, and Pick 1999|), the 



authors have proposed a framework within which agents may be built on top 
of an existing body of legacy code, and/or on top of speciaHzed data structures 
appropriate for the intended functionality of the agent being built. 

However, there are an increasing number of applications where agents are 
uncertain about what is true in their state (or environment). Such situations 
occur all the time in image identification programs, in programs that predict 
future events (such as battlefield events, stock market events, etc.), and in 
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scenarios where an agent a attempts to predict what an agent b will do. 

In this paper, we first introduce the concept of a probabilistic code call, 
which is a mechanism to describe uncertain application program interfaces for 
arbitrary functions over arbitrary data types. Based on this concept, we define 
probabilistic agent programs — sets of rules that encode the operating principles 
of an agent. Such rules encode the probabilistic conditions under which an agent 
is obliged to take some actions, permitted to take some actions and/or forbidden 
to take some actions. 

We then provide two broad classes of semantics for such "probabilistic agents." 
In the first class of semantics, actions that are permitted, obligatory or done, 
must have preconditions that are true with 100% probability in the current 
agent state. In the second class of semantics (which use probabilistic variants 
of Kripkc structures), the actions that are permitted, obligatory or done, must 
have preconditions that are true with at least a given probability. This latter 
class of semantics allows reasoning by cases. We provide complexity arguments 
showing that though the second family of semantics is perhaps epistemologically 
more appealing than the first, the second family of semantics is also computa- 
tionally more complex. 

Finally, the paper includes algorithms to compute the semantics of proba- 
bilistic agent programs, as long as such programs are negation free. 

Future work on probabilistic agent programs will focus on computing the 
semantics of paps that contain negation. A detailed study of computational 
complexity is also envisaged. We are also interested in identifying polynomially 
computable fragments of paps and implementing them on top of the current 
IMPACT implementation. Last, but not least, IMPACT has been used in a 
battlefield monitoring application where there is considerable uncertainty in 
predicting tactical enemy movements. We hope to build an application of paps 
addressing this problem, once the implementation of paps is complete. 
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A Agent Programs without Uncertainty 

The following definitions are taken from ( Eiter, Subrahmanian, and Pick 1999| ). 



A.l Feasible, Rational and Reasonable Semantics 

Definition A.l (Status Set) A status set is any set S of ground action status 
atoms over S. For any operator Op G {P, Do , F, O, W}, we denote by Op{S) 
the set Op{S) = {a \ Op{a) G S} . 
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Definition A. 2 (Deontic and Action Consistency) A status set S is called 
deontically consistent, if, by definition, it satisfies the following rules for any 
ground action a: 

• IfOae S, then Wa ^ S 

• IfPa e S, then Fa ^ 5 

• If Pa G S, then Og \= 3*Pre{a), where 3*Pre{a) denotes the existential 
closure of Pre{a), i.e., all free variables in Pre{a) are governed by an 
existential quantifier. This condition means that the action a is in fact 
executable in the state Os. 

A .status set S is called action consistent, if S, Os ^ AC holds. 

Besides consistency, we also wish that the presence of certain atoms in S 
entails the presence of other atoms in S. For example, if Oa is in S, then we 
expect that Pa is also in 5*, and if Oa is in 5*, then wc would like to have Do a 
in S. This is captured by the concept of deontic and action closure. 

Definition A. 3 (Deontic and Action Closure) The deontic closure of a sta- 
tus S, denoted D-C1(5), is the closure of S under the rule 

If Oa G S, then Pa G S 

where a is any ground action. We say that S is deontically closed, if S = 
D-C1(5) holds. 

The action closure of a status set S, denoted A-Cl(S'), is the closure of S 
under the rules 

If Oa G S, then Do a G S 

If Do a G S*, then Pa G S 

where a is any ground action. We say that a status S is action-closed, if S = 
A-C^S*) holds. 

The following straightforward results shows that status sets that are action- 
closed are also deontically closed, i.e. 

Definition A. 4 (Operator Appp^c'g(S')) Suppose V is an agent program, and 
Os is an agent .state. Then, App-pQ^ (S) is defined to be the set of all ground 
action status atoms A such that there exists a rule in P having a ground instance 
of the form r : A ^ ii, . . . , L„ such that 

1- B+^{r) C S and -.^-.(r) n 5 = 0, and 

2. every code call x G Btd''') succeeds in Os, and 

3. every code call x G '^■Bcd''') '^^^^ ^''^ succeed in Os, and 
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4-. for every atom Op{a) G B~^{r) U {A} such that Op £ {P,0,Do}, the 
action a is executable in state Os- 

Note that part (4) of the above definition only appUes to the "positive" 
modes P, O, Do . It does not apply to atoms of the form Fa as such actions are 
not executed, nor does it apply to atoms of the form Wa, because execution of 
an action might be (vacuously) waived, if its prerequisites are not fulfilled. 

Our approach is to base the semantics of agent programs on consistent and 
closed status sets. However, we have to take into account the rules of the 
program as well as integrity constraints. This leads us to the notion of a feasible 
status set. 

Definition A. 5 (Feasible Status Set) Let V be an agent program and let Os 
be an agent state. Then, a status set S is a feasible status set for V on 0$, if 
the following conditions hold: 

(SI): (closure under the program rules) App-p^Osi^) ^ ^j 

(52) (deontic and action consistency) S is deontically and action consistent; 

(53) (deontic and action closure) S is action closed and deontically closed; 

(54) (state consistency) O'g \= TC, where O'g ~ apply(Do{S),Os) is the 
state which results after taking all actions in 00(5*) on the state Og. 



Definition A. 6 (Groundedness; Rational Status Set) A status set S is 
grounded, if there exists no status set S' ^ S such that S' ^ S and S' sat- 
isfies conditions (5*1) -(5*3) of a feasible status set. 

A .status set S is a rational status set, if S is a feasible status set and S is 
grounded. 

Definition A. 7 (Reasonable Status Set) Let V be an agent program, let 
Os be an agent state, and let S be a status set. 

1. LfV is a positive agent program, then S is a reasonable status set for V 
on Os, if and only if S is a rational .status set for V on Os- 

2. The reduct ofV w.r.t. S and Os, denoted by red^{V, Os), is the program 
which is obtained from the ground instances of the rules in V over Os as 
follows. 

(a) First, remove every rule r such that B^g{r) n S* ^ 0; 

(b) Remove all atoms in B~g{r) from the remaining rules. 

Then S is a reasonable status set for V w.r.t. Os, if it is a reasonable 
status set of the program red^ (P, Os) with respect to Os- 
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B Proofs of Theorems 

Proof: (of Proposition ^.15) 



1. Suppose Do a S VS. Then, as VS is feasible, we know tliat VS = 
A-C\{VS), and lience Pa G VS. As VS is feasible, and hence deonti- 
cally consistent, the third condition of deontic consistency specifies that 
a's precondition is true in state Os. 

2. This follows immediately because as VS is feasible, we have VS = A-Cl {VS) . 
The second condition defining A-C1(7-'S'), when written in contrapositive 
form, states that Pa ^ VS implies that Do a ^ VS. 

3. As VS is feasible, VS = A-C\{VS). The first condition specifying A-C1(P5) 
allows us to infer that Oa e VS implies that Do a £ VS. The result fol- 
lows immediately from part (1) of this proposition. 

4. From the above argument, as VS — A-Cl('PS'), we can conclude that 
Oa e VS implies that Pa G VS. By the deontic consistency requirement, 
Fa i VS. 



Proof: (of Theorem 5.16) 



In order to show that a reasonable probabilistic status set VS of VV is a rational 
status of VV, we have to verify (1) that VS is a feasible probabilistic status set 
and (2) that VS is grounded. 

Since VS is a reasonable probabilistic status set of VV, it is a rational 
probabihstic status set of VV' ~ red^^{VV, Os), i.e., a feasible and grounded 
probabilistic status set of VV' . Since the conditions {V S2)-{V SA) of the defi- 
nition of feasible probabilistic status set depend only on VS and Os but not on 
the program, this means that for showing (1) it remains to check that {VS\) 
(closure under the program rules) is satisfied. 

Let thus r be a ground instance of a rule from VV. Suppose the body 
B{r) of r satisfies the conditions 1.-4. of {VSl). Then, by the definition of 
red^^{VV, Os), we have that the reduct of the rule r, obtained by removing all 
literals of B~g{r) from the body, is in VV' . Since VS is closed under the rules 
of VV' , we have H{r) £ VS. Thus, VS is closed under the rules of VV, and 
hence (PS'l) is satisfied. As a consequence, (1) holds. 

For (2), we suppose VS is not grounded, i.e., that some smaller VS' C VS 
satisfies {V S1)-{V Si) for VV, and derive a contradiction. liVS' satisfies [VSl) 
for VV, then VS' satisfies {VSl) for VV' . For, if r is a rule from VV' such 
that 1.-4. of {VSl) hold for VS' , then there is a ground rule r' of VV such 
that r is obtained from r' in the construction of red^^ {VV, Os) and, as easily 
seen, 1.-4. of {VSl) hold for VS'. Since VS' satisfies {VSl) for VV, we have 
H{r) £ VS'. It follows that VS' satisfies {VSl) for VV'. Furthermore, since 
{VS2) and {VS3) do no depend on the program, also {VS2) and {VS3) are 
satisfied for VS' w.r.t. VV' . This means that VS is not a rational probabilistic 
status set of VV' , which is the desired contradiction. 
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Thus, (1) and (2) hold, which proves the resuh. 



Proof: (of Theorem 5.19) 



By definition of rationahty, we know that if VS is a rational status set of W 
then it must be a feasible probabilistic status set as well. 

Suppose VP has a feasible probabilistic status set. Then the set of all 
feasible probabilistic status sets of W on Os has a non-empty set of inclusion- 
minimal elements. Indeed, from the grounding of the probabilistic agent pro- 
gram, we can remove all rules which violate the conditions 2.-4. of the operator 
App-p-p Og (VS), and can remove literals involving code calls from the remaining 
rules. Moreover, the deontic and action closure conditions can be incorporated 
into the program via rules. Thus, we end up with a set T of propositional 
clauses, whose models are feasible probabilistic status sets of W. Since VP 
has a feasible probabihstic status set, T has a model, i.e., an assignment to the 
propositional atoms which satisfies all clauses in T. Now, each satisfiable set 
of clauses in a countable language posseses at least one minimal model (w.r.t. 
inclusion, i.w., a C-minimal set of atoms is assigned the value true); this can be 
shown applying the same technique which proves that every such set of clauses 
can be extended to a maximal satisfiable set of clauses. Thus, T has at least 
one minimal model. As easily seen, any such model is a minimal feasible prob- 
abilistic status set of VP. 

Suppose now VS' is one of the minimal feasible probabilistic status sets of 
VP on Os- Then (as we show below) VS' is grounded, and hence a rational 
probabilistic status set. 

To show that VS' is grounded, we need to show that VS' satisfies conditions 
{VS 1)-{VS 3) of the definition of feasible probabilistic status set — this is true 
because VS' is feasible. In addition, we need to show that no strict subset VS* 
of VS satisfies conditions (VS 1)-(VS 3). 

Suppose there is a strict subset VS* of VS satisfying conditions (VS 1)- 
{VS 3). Then, as IC = 0, VS* also satisfies condition (VS 4) of the definition 
of feasibility, and hence VS* is a feasible probabilistic status set. But this 
contradicts the inclusion minimality of VS' , and hence, we may infer that VS' 
has no strict subset VS* oiVS satisfying conditions (VS l)-(VS 3). Thus, VS' 
is grounded, and we are done. | 



Proof: (of Theorem |7.6|) 



For each random variable Vi — (Xi , pi ) returned by some ground code call 
condition in the probabilistic state O^, let us define its normalized version V/ = 
(a;, p^) where: 

Xl = {x\xe X, and p,(x) > 0} U {e | J2xex^ P^i^) < 1); 

r p,{x) iix£Xi\{e} 

P^(^) - I 1 _ J2,^^^ P^ix) if X = £ and e e A,' 

i.e., we delete the zero-probability elements and add the extra one e (which 
stands for "none of the above" ) whenever the distribution pi is incomplete. 
Now we can see that each tuple x = {xi, . . . ,Xn) in the Cartesian product 
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X' — X'l X • • • X X'^ corresponds to a distinct compatible state O w.r.t. O^ . 
In O, a ground code call returns an object o iff in the probabilistic state O^ it 
returns a variable Vi such that Xi — o. Let associate to each state O of this kind 
the value 

P*{0) = p'^{xi) ■ ■ ■ p'^{Xn) 

andsetp*(C') = for all the other states. We can easily verify that (COS(C'p), p*) 
is a compatible probabilistic Kripke structure for O^: for each random variable 
Vi returned in state O^ and each object o G X^ if pi{o) = then o ^ X' , so it 
could appear only in the zero-probability states; otherwise: 

n 
oeO xGlC,Xi=oJ=0 xG'7P',Xi=oJ¥'i 

Both cases satisfy the condition for compatibility. Finally, it is easy to verify 
that (COS(C'P), p*) is really a probabilistic Kripke structure: 

n 



Proof: (of Theorem 7.7) 



Let us consider the compatible Kripke structure described in the proof of Propo- 
sition 7.6 on page ^, and let us assume that Vi and V2 are the variables required 



in the thesis. The corresponding completed versions V( and V^' will then contain 
at least two non zero-probability objects (one of them could be the extra object 
e), respectively ai,5i and 02,62- Now let choose an arbitrary real number 6 
such that: 

0<S< niin lp'i(a:)pi(y)} 

x'£{ai,bi},ye{a2M} 

We can build a Kripke structure (COS(C'p), p*^), where p* is defined in the same 
way as p* but replacing p'i{xi)p2{x2) by (f){xi,X2), which in turn is defined in 
the following way: 

{Pi{xi)p2{^2) -S if (xi,a;2) G {(ai, a2), (&i, 62)} 
p'i{xi)p'2{x2) + S if (xi,a::2) £ {(01,62), (61,02)} 
Pi(a;i)p2(a;2) otherwise 

It is easy to verify that it is a compatible Kripke structure. Since S can be 
arbitrarily chosen within a non-point interval, we can obtain an infinite number 
of distinct compatible Kripke structures. | 



Proof: (of Theorem 8.12) 



(=^) Suppose VS = lfp{S-p'P,Os) ^ rational probabilistic status set of W on 
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Os- Then, VS is feasible by definition of rational probabilistic status set. By 



Lemma 5. IS, VS is a pre-fixpoint of S-p-p^Os- Since S-p-p^Os i^ monotone, it 
has by the Knaster-Tarski Theorem a least pre-fixpoint, which coincides with 
IfpiSpp^Os) (see ( |Apt f990| ; ^loyd 1987[ )). Thus, lfp{Spp^Os) ^ 'PS. Clearly, 



lfp{Spp^Os) satisfies (VSl) and {VSS); moreover, lfp{Spp,Os) satisfies {VS2), 
as VS satisfies {VS2) and this property is hereditary. By the definition of 
rational probabilistic status set, it follows lfp{Spp,Os) — ^S. 

(<=) Suppose VS — lfp{Spp^Os) is a feasible probabilistic status set. Since 
every probabilistic status set VS' which satisfies ('P5'1)-('PS'3) is a pre-fixpoint 
of S-p-p^Os and lfp{Sppog) is the least prefix point, VS C VS implies VS = 
VS' . It follows that VS is rational. 

Notice that in case of positive programs, lfp{Spp^o^) always satisfies the 
conditions (VSl) and {VS3) of a feasible probabilistic status set (i.e., all closure 
conditions), and thus is a rational probabilistic status set if it satisfies {VS2) and 
{VSA), i.e., the consistency criteria. The uniqueness of the rational probabihstic 
status set is immediate from the previous theorem. | 
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